fix: apply logos-nix's fetchCargoVendor User-Agent overlay in mkPkgsWith - #220
Closed
danisharora099 wants to merge 1 commit into
Closed
fix: apply logos-nix's fetchCargoVendor User-Agent overlay in mkPkgsWith#220danisharora099 wants to merge 1 commit into
danisharora099 wants to merge 1 commit into
Conversation
…package sets crates.io returns 403 to the pinned nixpkgs' UA-less fetchCargoVendor helper, so every module release's cargo vendoring fails (logos-co#159). logos-nix#6 carries the upstream fix as lib.overlays.fetchCargoVendorUserAgent, but this repo does its own `import nixpkgs` in mkPkgsWith, so the overlay never reached module builds. Thread it in there, guarded so a logos-nix pin that predates the attribute still evaluates (no-op until the lock bump). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ezs1fWormzPw9QqU89oAFU
Contributor
|
I'm guessing you need to |
dlipicar
added a commit
that referenced
this pull request
Sep 8, 2026
mkPkgsWith was a bare `import nixpkgs { overlays = extraOverlays; }`, so every
module's package set missed logos-nix's crates.io fixes. crates.io returns 403
to any User-Agent starting "curl/" -- exactly what nixpkgs' fetchurl sends --
so a Rust module 403s on any crate the org cache misses.
mkLogosModule vendors with `cargoLock`, i.e. through importCargoLock, NOT
fetchCargoVendor. #220 wires only the UA overlay, which covers the other
fetcher; it is not the one module crates go through.
Take logos-nix's `lib.nativeOverlays` list rather than naming entries. Naming
them is how the importCargoLock fix reached master applying to nothing, and a
third overlay would repeat it. The pinned logos-nix moves to master for the
export (nixpkgs stays at e9f00bd8 and the lock keeps all 756 nodes, so nothing
else in the closure shifts).
An old logos-nix throws rather than silently applying nothing. #220's
`lib.optional (logos-nix ? ...)` shape degrades to a no-op on a stale pin and
re-opens the 403 with no diagnostic, which is the failure mode this whole chain
exists to close. `logos-nix == null` still yields [ ] and is unaffected.
Measured on checks.aarch64-darwin.rust-native-dep, reading `urls` off the crate
FODs in the realised closure:
before 15 crate fetches, all on https://crates.io/api/v1/crates
after 12 moved to https://static.crates.io/crates
NOT SUFFICIENT ON ITS OWN. The other 11 belong to logos-lidl-gen, built by
logos-rust-sdk, which does its own bare `import nixpkgs` (flake.nix:69 and
:196) and so is untouched by this. A module build still reaches the 403
endpoint for those until logos-rust-sdk gets the same wiring -- and that is the
repo whose doc-tests run started this.
Blast radius: the only derivations this change touches are fixed-output ones
(the crate tarballs and the *-vendor-staging FODs) plus one new
fetch-cargo-vendor-util-ua helper -- 46 of 1284, every root a FOD, so
hashDerivationModulo absorbs them and none propagates. Module output paths do
move, for the unrelated pre-existing reason that LOGOS_MODULE_BUILDER_ROOT
embeds the builder's source path: appending a single no-op comment to
lib/common.nix moves them exactly the same way.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
|
Heads-up: I've opened #233, which I think subsumes this — happy to close whichever you prefer. The reason for a second PR rather than a review comment is that the overlay wired here is the wrong fetcher for module builds. Measured on Two other things #233 changes that came out of the same investigation:
Worth saying plainly: neither PR actually unblocks Rust module builds on its own. 11 of the crate fetches in that closure belong to |
Contributor
|
closed in favour of #233 |
dlipicar
added a commit
that referenced
this pull request
Sep 8, 2026
mkPkgsWith was a bare `import nixpkgs { overlays = extraOverlays; }`, so every
module's package set missed logos-nix's crates.io fixes. crates.io returns 403
to any User-Agent starting "curl/" -- exactly what nixpkgs' fetchurl sends --
so a Rust module 403s on any crate the org cache misses.
mkLogosModule vendors with `cargoLock`, i.e. through importCargoLock, NOT
fetchCargoVendor. #220 wires only the UA overlay, which covers the other
fetcher; it is not the one module crates go through.
Take logos-nix's `lib.nativeOverlays` list rather than naming entries. Naming
them is how the importCargoLock fix reached master applying to nothing, and a
third overlay would repeat it. The pinned logos-nix moves to master for the
export (nixpkgs stays at e9f00bd8 and the lock keeps all 756 nodes, so nothing
else in the closure shifts).
An old logos-nix throws rather than silently applying nothing. #220's
`lib.optional (logos-nix ? ...)` shape degrades to a no-op on a stale pin and
re-opens the 403 with no diagnostic, which is the failure mode this whole chain
exists to close. `logos-nix == null` still yields [ ] and is unaffected.
Measured on checks.aarch64-darwin.rust-native-dep, reading `urls` off the crate
FODs in the realised closure:
before 15 crate fetches, all on https://crates.io/api/v1/crates
after 12 moved to https://static.crates.io/crates
NOT SUFFICIENT ON ITS OWN. The other 11 belong to logos-lidl-gen, built by
logos-rust-sdk, which does its own bare `import nixpkgs` (flake.nix:69 and
:196) and so is untouched by this. A module build still reaches the 403
endpoint for those until logos-rust-sdk gets the same wiring -- and that is the
repo whose doc-tests run started this.
Blast radius: the only derivations this change touches are fixed-output ones
(the crate tarballs and the *-vendor-staging FODs) plus one new
fetch-cargo-vendor-util-ua helper -- 46 of 1284, every root a FOD, so
hashDerivationModulo absorbs them and none propagates. Module output paths do
move, for the unrelated pre-existing reason that LOGOS_MODULE_BUILDER_ROOT
embeds the builder's source path: appending a single no-op comment to
lib/common.nix moves them exactly the same way.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Every module release's cargo vendoring dies with
crates.io ... 403(#159): the pinned nixpkgs'fetchCargoVendorhelper sends no User-Agent. logos-co/logos-nix#6 carries the upstream fix (NixOS/nixpkgs#512735) aslogos-nix.lib.overlays.fetchCargoVendorUserAgenton the current pin, no Qt bump — per the 2026-08-27 plan on logos-co/logos-nix#5 (logos-co/logos-nix#5 (comment)) and the 2026-08-12 maintainer ask that the fix live in logos-nix.But logos-nix's overlay only reaches package sets built through logos-nix's own helpers. This repo does its own
import nixpkgsinlib/common.nixmkPkgsWith, so it never applied here. This is the one consumer line that hooks it up.What
lib/common.nixmkPkgsWith: prependlogos-nix.lib.overlays.fetchCargoVendorUserAgentto the native package set's overlays. Guarded withlogos-nix ? lib.overlays.fetchCargoVendorUserAgent, so on the current lock (which predates the attribute) it is a no-op and evaluation is unchanged — the fix switches on with the lock bump. Not applied to thex86_64-windowsset (mkWindowsPkgsowns that;nixpkgs-windowsalready has the upstream fix).No
flake.lockchange. Merge order: after logos-co/logos-nix#6, then anix flake update logos-nixlock bump here (separate commit/PR) — only then do module releases pick this up, via theirlogos-module-builderinput.Verified (aarch64-darwin,
--override-input logos-nix github:logos-co/logos-nix/aac5338e= #6's head)checks.aarch64-darwin.rust-native-depinstantiates; the module's own crate compile (rust_native_dep-1.0.0.drv) now resolves its toolchain through the overlayed set — itscargo-auditable/cargo-c/rav1e-vendor-stagingFODs referencefetch-cargo-vendor-util-ua, whose built helper setsUser-Agent: nixpkgs-fetchCargoVendor/2 (...)and fetches fromstatic.crates.io. Without the override, every staging drv in that closure references the unpatched helper.rustPlatform.fetchCargoVendor(bitflags 2.9.4) built from this repo'scommon.mkPkgs "aarch64-darwin": without the overlay →Status code: 403(reproduces Release builds broken ecosystem-wide: crates.io returns 403 to fetchCargoVendor (UA-less python fetch) #159); with it → fetches fromstatic.crates.ioand produces thecargo-deps-vendoroutput (bitflags-2.9.4/+.cargo/).checks.aarch64-darwin.*still evaluate;rust-native-dep's drvPath is unchanged (guard is a no-op).Not proven: the
rust-native-depfixture has no external crates (its vendor dir is local), so a full run of that check does not exercise a crates.io download — hence the one-crate vendor above. Staging drvs reached vialogos-lidl-gen(logos-rust-sdk's own nixpkgs set) stay unpatched; those are cache-served upstream tools, not a module's crates, and are out of this repo's hands. Linux not run.🤖 Generated with Claude Code
https://claude.ai/code/session_01Ezs1fWormzPw9QqU89oAFU